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Related Appeals and Interferences : 

None. 



Status of Claims : 

Claims 17-41 are pending in the application. Claims 17-41 currently stand rejected 
under 35 U.S.C. § 103. 

Status of Amendments : 

On February 26, 2003 Applicant filed amendments to address objections made to 
claim 17, 23, 24, 30 and 36. These amendments were entered in the advisory action mailed 
March 28, 2003. 

Summary of the Invention : 

Applicant's invention provides for methods relating to evaluating objects having 
three dimensional (3D) features. The invention provides in part a method to increase through-put 
of a 3D inspection system. The method may utilize two dimensional (2D) image data to pre- 
qualify the validity of 3D image data and potentially eliminate the need to evaluate the three- 
dimensional data. See specification at p. 6, 11.4-7. 

In particular, the 2D image, including its 2D data, will include 2D data 
characteristic of 3D features. If those 3D features are not in the expected locations in the 2D 
image the part can be rejected. See specification at p. 6, 11.9-12. The method of the present 
invention also provides processing efficiencies by locating data characteristic of 3D features in the 
2D image and utilizing those locations to reduce the data processed within the 3D image. 



Preferably, the method evaluates integrated circuit packages (IC packages) where 
the packages include three dimensional features. See specification at p. 5, 11.17-19. Such IC 
packages may include a ball grid array 20. A sensor processor 10 including a camera 1 1 captures 
light reflected from balls 22 and ultimately converts the light value to pixel values. See 
specification at p. 6, line 30-p.7, line 11. The sensor output includes a 3D representation of the 
3D features of the BGA. See specification at p. 8, 11.8-10. This 3D information includes height 
data. See specification at p. 8, 11. 10-1 1. Thus, the 3D image includes x, y, and z components. 
Sensor 10 may be configured to capture both a 3D image and a 2D image of the three dimensional 
features on the IC package. See specification at p. 9, 11. 27-30. See p. 1 1, 11.5-10. 

The present invention allows for calibration between data measurements and real 
world coordinates. See specification at p. 1 1, 11.20-22. The calibration further establishes a 
relationship between the 2D image and the 3D image including appropriate scaling and correction 
factors. See specification at p. 1 1, 11.22-25. The calibration process may be accomplished by 
performing processing on a machine fixture 50 which contains specifically defined survey points. 
See specification at p. 1 1, 11.25-27. 

The invention provides a method to correct for geometric distortion. Geometric 
distortion is an artifact associated with the use of oblique laser/sensor combinations where 
distortion in the measurements is caused by virtue of the displacement as a reflected laser beam 
due to the height of the object being scanned. See specification at p. 14, 11. 21-27. Figure 12 of 
the application illustrates that z height causes images to be sensed by the sensor off-center. See 
specification at p. 1 5, 11. 1-2. As shown in Figure 14, the sensor attempts to measure the altitude of 
the object to position xo, but laser beam 16 is intercepted by the object at 13' at position xl with 
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an altitude z yielding the measured displacement of d. The measure d is proportional to the 
altitude of the object at 13' at the point measured and is a measure of the displacement from the 
actual location of object 13'. See specification at p. 15, 11.13-18. The present invention provides 
multiple ways to correct geometric distortion when evaluating data in a 3D image. See 
specification at p. 1 5, 11.20-21 . Theses methods involve utilization of the equations listed on page 
15 of the specification to compare the data against a similarly distorted template. 

The present invention also includes a description of efficient 3D image processing 
where 2D data (or 2D image) is processed to locate 3D features. See specification at p. 16, 11.21- 
22. The invention can first pre-qualify 3D features by examining a 2D image in a pre-processing 
operation to determine whether data characteristic of 3D features can be located in expected 
locations as well as the use of 2D data to guide the processing of 3D data to save processing 
resources. See specification at p. 16, 1.20-p.l7, 1.8. Use of 2D data is described in greater detail 
on page 1 8 of the specification with reference to Figure 17. The 2D images are used as a first 
pass, or pre-processing step to preliminarily identify and locate data characteristics and 3D 
features before processing 3D images to make, for example, measurements of coplanarity. See 
specification at p. 18, 11. 26-29. 

The location of the 3D features as determined by the 2D processing is passed to 
the 3D processing routines such that the 3D image can be processed at the location of the 3D 
features as provided by the 2D processing. See specification at p. 19, 11.29-3 1 . In the preferred 
embodiment locating 3D features in the 3D image set, based on the 2D image, can utilize the 
2D/3D transform acquired from the calibration process to precisely locate data which actually 
represents 3D features. See specification at p. 20, 11.29-30. 



Issues on Appeal : 

Issue: Whether the disclosure of Jackson et al. (U.S. Patent No. 5,652,658) is 
sufficient to establish a prima facie case of obviousness against claims 17-41 . 

Grouping of Claims : 

Applicant asserts that claims 17-41 do not stand or fall together. The basis for this 
position is set forth in greater detail under the subsequent section entitled Argument. Claims 17- 
24 and 27 will stand and fall together depending on the disposition of claim 17. Claims 25, 26, 
and 30-33 will stand or fall together depending on the disposition of claim 30. Claims 28, 34 and 
36-41 stand or fall depending on the disposition of claim 36. Claim 29 stands independently. 
Claim 35 stands independently. 

Argument : 

The Examiner has rejected claims 17-41 under 35 U.S.C § 103. In particular, the 
Examiner has rejected claims 17-22, 25, 26, 28, 30-32, 34, and 36-39 under 35 U.S.C § 103(a) 
as being unpatentable over Jackson et al. (U.S. Patent No. 5,652,658.) The Examiner rejected 
claims 23 and 24 as being unpatentable over Jackson et al. as applied to claim 22 and further in 
view of Roy et al. (U.S.Patent No. 5,402,505). The Examiner rejected claims 27, 33, and 40 
under 35 U.S.C § 103(a) as being unpatentable over Jackson as applied to claims 17, 30 and 39 
in further view of Chen et al. (U.S.Patent No. 5,028,799.) The Examiner rejected claims 29 and 
35 under 35 U.S.C § 103(a) as being unpatentable over Jackson as applied to claims 28 and 34 in 
further view of Williams (U.S.Patent No. 4,801,207). The Examiner rejected claim 41 under 35 
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U.S.C. § 103(a) as being unpatentable over Jackson in view of Chen as applied to claim 40 and 
further in view of Williams. The Examiner also objected to claims 27, 23, 24, 30 and 36 based on 
specified formalities. 

Applicant responded to the Final Office Action mailed January 2, 2003. In an 
advisory action mailed March 28, 2003, the Examiner withdrew the objections to the claims but 
maintained the 35 U.S.C. § 103 rejection of claims 17-41. 

A. The Examiner's Rejection of Claims 17-24 and 27 

With respect to the rejection of claims 17-24 and 27 the Examiner contends that 
Jackson et al. discloses a method of evaluating comprising acquiring a 2D address and pixel 
intensity and a 3D image with an address and altitude (citing col.5, 11.35-39 of the Jackson patent) 
and rejecting an IC package based on altitude (citing col.6, 11.12-15), co-planarity (citing col.3, 11. 
15-22) and shape of spheres (citing col.4, 11.5-10) with the template or fixture and gray scale 
images (citing col.5, 11.63-67). The Examiner concedes that Jackson does not disclose processing 
the 2D image to identify 3D features and processing the 3D image at those features to determine 
the altitude of those features in one embodiment of Jackson. The Examiner goes on to state that 
Jackson teaches processing the 2D image to identify 3D features and processing the 3D image at 
those features to determine the altitude of those features in another particular embodiment (citing 
col. 6, 11.26-35 and 40-44). The Examiner states that it would have been obvious to one having 
ordinary skill in the art at the time the invention was made to have the processing of Jackson in 
one embodiment with the method of Jackson. The Examiner states that because the processes 
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were recognized as equivalents at the time the invention was made, one of ordinary skill in the art 
would have found it obvious to substitute one process for another. 

Claim 17 is directed toward a method of evaluating an IC package involving 
acquiring a 2D image characteristic of a portion of the IC package, processing a 2D image to 
identify a plurality of addresses which are characteristic of 3D features, processing the 3D image 
only at those addresses which correspond to 2D addresses characteristic of 3D features and 
rejecting the IC package if the altitude of the 3D features fall outside predetermined boundaries. 
The method of claim 17 provides for an efficient processing of 3D data which reduces 
unnecessary processing of features which do not contain 3D data. 

To establish a prima facie case of obviousness of obviousness, three basic criteria 
must be met. First there must be some suggestion or motivation either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the art to combine 
reference teachings. Second, there must be a reasonable expectation of success. Finally, the prior 
art references when combined must teach or suggest all the claimed limitations, MPEP § 2142. 
The teaching or suggestion to make the claimed combination and the reasonable expectation of 
success must be found in the prior art and not be based on the Applicant's disclosure. In re Vaeck 
947 F.2d 488 (Fed.Cir.1991). 

Under the first criteria of the prima facie case of obviousness, the recently decided 
case of In re Lee 277 F.3d 1338 (Fed. Cir. 2002) clearly defines how suggestion and motivation are 
determined and how the knowledge generally available to one of skill in the art is found. The 
Federal Circuit In re Lee reviewed a decision from the Board of Patent Appeals and Interferences 
in the United States Patent and Trademark Office. The Examiner and the Board agreed that the 
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invention "Would have been obvious to one of ordinary skill in the art since the demonstration 
mode is just a programmable feature which can be used in many different device[s] for providing 
automatic introduction by adding the proper programming software," and that "another 
motivation would be that the automatic demonstration mode is user friendly and it functions as a 
tutorial." Id at 1341 . However, in this case, the Federal Circuit made it abundantly clear that the 
Board's and the Examiner's conclusory statements did not adequately address the issue of 
motivation to modify a reference or motivation to combine references. The factual question of 
motivation is material to patentability, and could not be resolved on subjective belief and unknown 
authority. Id at 1343. It is improper, in determining whether a person of ordinary skill in the art 
would have been led to this combination of references, simply to "[use] that which the inventor 
taught against its teacher." W.L.Gore v. Gorlocklnc. 721 F.2d 1540, 1553 (Fed.Cir.1983). The 
court In re Lee went on to state that the "common knowledge and common sense" on which the 
Board relied in rejecting Lee's application are not the specialized knowledge and expertise 
contemplated by the administrative procedure act. Conclusory statements such as those here 
provided do not fulfill the agency's obligation. 277 F.3d 1342. 

The Examiner has mis-characterized Jackson. Jackson describes a grid array 
inspection system which identifies height and position data pertaining to both ball grid arrays and 
pin grid arrays. See Jackson col.4, 11.6-8. Jackson states that the grid array in section system 10 
comprises a 3D scanner 16. See Jackson col.4, 11. 28-29. The patent goes on to describe that this 
3D data, which covers multiple pins or balls, is "provided at high rates. . .[and] each line provides 
512 points of 8 bit height data as well as 8 bits of intensity (gray scale) data used to determine the 
position of the pin or ball grid in the direction of the electronic scan." See Jackson at col.4, 11.56- 
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61. At this point Jackson does not yet state how the 3D features are located. The patent then 

goes on to state that M [s]ince the 3D scanner 16 scans only in one dimension and the grid arrays 

12 are larger than the scan width of the three dimensional scanner 16, the three dimensional 

scanner 16 must be moved to provide full coverage of the grid array 12." See Jackson at col. 5, 11. 

7-10. As noted at col.5, 11.35-42, this 3D data provides both intensity data which is 2D and height 

data which is 3D. With reference to col.5, line 55, Jackson describes how determination of pin 

positions (i.e. the 3D features) is accomplished. Jackson states that a user identifies the type of 

pin grid array being inspected and enters an identification code into the computer. See Jackson at 

col.5, 11.56-60. As illustrated in Fig. 5 A the computer matches the identification code to a course 

alignment grid from a data file. Id. Jackson then goes on to state with reference to Fig.5B that 

the course position grid consists of a block of predetermined dimensions containing pixel data 

representing the height measurement for each pixel. Thus Jackson does not use 2D addresses to 

process 3D image data and indeed Jackson never obtains 2D address corresponding to 3D 

features. At col. 6, 11. 1-1 1 Jackson specifically states that: 

the individual values representing the height data for that pixel 86 facilitates the 
determination of the height and physical location of each pin 84. Figure 5C 
illustrates the location of the tip 88 of a pin 84. Once the course position of a pin 
84 is located the algorithm performs a fine search 90 of the image for pixels 86 
which represent the tip 88 of the pin 84. The height value for the tip 88 of the pin 
84 will be significantly greater than the surrounding height values representing the 
bottom surface 92 of the grid array package 80. Once the height and position data 
is determined coplanarity and regression plots can be developed and analyzed. 

See Jackson at col. 6, 11. 1-1 1 (emphasis added). Unlike the Examiner's belief, Jackson uses the 3D 

image data to locate the 3D features. 
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Jackson provides no teaching whatsoever to locate data characteristic of 3D data within a 2D 
image. Rather, in Jackson the 3D data is processed to determine the location of the 3D data. 

Jackson goes on to describe multiple techniques to detect compound bends. 
Jackson describes alternate methods to detect compound bends. One alternate method to identify 
compound bends utilizes 2D intensity data. This method is described at col.6, 11.40-44. In 
particular the invention states that "using histogram techniques the algorithm identifies a 
compound bend when there is a relatively large number of dark pixels in the area of interest." 

Here, the Examiner again mis-characterizes Jackson in the rejection. Based on the 
above description of Jackson it appears that the Examiner is picking and choosing technical 
concepts from Jackson to provide the foundation for the rejection of claims 17-24 and 27. The 
Examiner rightly concedes that Jackson does not disclose processing a 2D image to identify 3D 
features and processing a 3D image at those features to determine the altitude of those features. 
The Examiner's reference to the second embodiment relative to compound bending to provide the 
teaching of processing a 2D image to locate 3D features is erroneous. Specifically, the additional 
aspect of Jackson describing that a compound bend can be identified through the use of 2D data 
does not relate at all to processing a 2D image to locate addresses associated with 3D data. 
Rather, Jackson suggests that the location of the pins are within a region of interest which is 
provided to the 2D processing and not the result of the 2D processing. 

In essence what the Examiner has done is merely filled in with the examiner's 
personal opinion relative to this claim in violation of Lee. Here the Examiner has pointed to no 
teaching where 2D data is processed to determine the location of data associated with 3D 
features. The Examiner provides no teaching other than their own personal belief as to how the 
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embodiment of Jackson to locate a compound curve could be modified such that data 
characteristic of 3D information could be identified. The Examiner's rejection of claims 17-24 and 
27 should be overturned. 

B. The Rejection of Claims 25, 26, and 30-33 

Claims 25, 26, and 30-33 stand or fall depending upon the disposition of claim 30. 
Claim 30 is different from claim 17 insofar as claim 30 adds the additional feature of comparing 
the 2D image against a 2D template and rejecting the quality of the IC package if the comparison 
reveals that the 2D image does not include 3D features in an expected configuration. 

Claim 30 should be allowable for all of the reasons stated above with respect to 
claim 17. However, with respect to the additional limitation, the Examiner's rejection is 
exceedingly ambiguous. The Examiner makes a single statement: that 3D data can be evaluated 
with a template (citing col. 5, 11.58-60). The Examiner's comments in the advisory action provide 
no further assistance. This has no bearing on whether a 2D image is used to locate and qualify a 
3D image. The method of claim 30 provides additional processing efficiencies insofar as a part 
can receive an early rejection based on the less computationally intensive evaluation of a 2D 
image rather than the computationally intensive evaluation of a 3D image. The Examiner has not 
established a prima facie case identifying this element and therefore the rejection of claims 25, 26, 
30-33 should be overturned. 
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C Claims 28, 34, and 36- 41 

Claims 28, 34, and 36- 41 stand or fall based on the disposition of claim 36. Claim 
36 stands rejected based solely on Jackson. Claim 36 includes the additional aspect of 
determining a correspondence between the addresses in the 2D image and the addresses in the 3D 
image by calibrating to a machine fixture. The Examiner's rejection of claim 36 based on Jackson 
is entirely silent with respect to reference to calibration to a machine fixture. For the reasons 
stated above with respect to claim 17, and for the failure to make a prima facie case with respect 
to the calibration to a machine fixture, Applicant submits that the Examiner's rejection of claims 
28, 34 and 36-41 should be overturned. 

D. The Examiner's Rejection of Claim 29 

Claim 29 includes all the limitations of claim 28 (determining a correspondence 
between the addresses in the 2D image and the addresses in the 3D image by calibrating to a 
machine fixture) and claim 17. For the reasons stated with respect to claim 17 and 36, claim 29 is 
allowable. Applicant further submits that the Examiner has violated the standards of hindsight to 
extract correction of a 2D 3D transformation relative to claim 29. Applicant requests that the 
rejection of claim 29 should be overturned. 

E. The Examiner's Rejection of Claim 35 

Claim 35 includes all the limitations of claim 29 and further includes the pre- 
screening of 2D information by comparing the 2D information to a template and rejecting the 
image if the 2D image does not include appropriate 3D content. For all the reasons stated above 
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with respect to claim 17, 30 and 36 this claim is allowable. Applicant requests that the rejection 
of claim 35 be overturned. 



Conclusion 

For the reasons stated above, it is respectfully submitted that Appellants' invention 
as set forth in claims 17-41 patentably define over the cited references and is not suggested or 
rendered obvious thereby. As such, it is respectfully submitted that the Examiner's final rejection 
of claims 17-41 is erroneously based and its reversal is respectfully requested. 

No oral hearing is requested. 

Appellant's attorney's check in the amount of $330.00 is enclosed to cover the 
Appeal Brief filing fee. 

This Appeal Brief is being filed in triplicate including one original and two copies. 
An Appendix listing all of the claims is attached. 

Respectfully submitted, 

YOUNG &BASILE, P C. 



Thomas E. B^jin 
Attorney for Applicant 
Registration No. 37,089 
(248) 649-3333 




3001 West Big Beaver Rd., Suite 624 
Troy, Michigan 48084-3107 



Dated: May 5, 2004 
TEB/sld 
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APPENDIX 

In the claims: 

17. A method for evaluating the quality of an IC package where the IC package 
includes a plurality of three dimensional features, the method comprising: 
acquiring a two dimensional image characteristic of a portion of the IC package, the two 
dimensional image defined by a plurality of pixels having at least an address and a pixel intensity; 

acquiring a three dimensional image characteristic of the portion of the IC 
package, the three dimensional image defined by a plurality of pixels having at least an address 
and an altitude; 

processing the two dimensional image to identify a plurality of addresses which are 
characteristic of three dimensional features; 

processing the three dimensional image only at those addresses which correspond 
to two dimensional addresses characteristic of three dimensional features, to determine the 
altitude of those three dimensional features, and; 

rejecting the IC package if the altitude of the three dimensional features fall outside 
predetermined boundaries. 

18 . A method as in claim 17 wherein the processing of the three dimensional image 
further comprises determining the shape of the three dimensional features and the IC package is 
rejected if the shape of the three dimensional feature falls outside a predetermined boundary. 

19. A method as in claim 18 wherein the three dimensional features are 
characteristic of spheres. 

20. A method as in claim 19 wherein the spheres are compared against templates. 

21 . A method as in claim 18 wherein the IC package is rejected if any three 
dimensional feature exceeds a predetermined value. 
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22. A method as in claim 18 wherein the IC package is rejected if the coplanarity 
value of a collection of three dimensional features is greater than a predetermined value, 

23. A method as in claim 22 where the coplanarity is determined by calculating 
planes of repose. 

24. A method as in claim 22 where the coplanarity value is determined by 
calculating a best fit plane using least squares. 

25. A method as in claim 17 further comprising comparing the two dimensional 
image against a two dimensional template and rejecting the quality of the 

IC package if the comparison reveals that the two dimensional image does not include three 
dimensional features in an expected configuration. 

26. (Previously Presented) A method as in claim 25 wherein the two dimensional 
image is a gray scale image and the two dimensional image is correlated against the two 
dimensional template. 

27. A method as in claim 17 wherein a pair of opposed lasers are used to obtain a 
first and second three dimensional image and the first and second three dimensional images are 
combined to obtain the three dimensional image. 

28. A method as in claim 17 further comprising determining a correspondence 
between the addresses in the two dimensional image and the addresses in the three dimensional 
image by calibrating to a machined fixture. 

29. A method as in claim 28 further comprising transforming the three dimensional 
image to remove geometric distortion. 
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30. A method for evaluating the quality of an IC package where the IC package 
includes a plurality of three dimensional features, the method comprising: 

acquiring a two dimensional image characteristic of a portion of the IC package, 
the two dimensional image defined by a plurality of pixels having at least an address and a pixel 
intensity; 

acquiring a three dimensional image characteristic of the portion of the IC 
package, the three dimensional image defined by a plurality of pixels having at least an address 
and an altitude; 

comparing the two dimensional image against a two dimensional template and 
rejecting the quality of the IC package if the comparison reveals that the two dimensional image 
does not include three dimensional features in an expected configuration; 

processing the two dimensional image to identify a plurality of addresses which are 
characteristic of three dimensional features; 

processing the three dimensional image only at those addresses which correspond 
to two dimensional addresses characteristic of three dimensional features, to determine the 
altitude of those three dimensional features, and; 

rejecting the IC package if the altitude of the three dimensional features fall outside 
predetermined boundaries. 

3 1 . A method as in claim 30 wherein the IC package is rejected if any three 
dimensional feature exceeds a predetermined value. 

32. A method as in claim 30 wherein the IC package is rejected if the coplanarity 
value of a collection of three dimensional features is greater than a predetermined value. 

33. A method as in claim 30 wherein a pair of opposed lasers are used to obtain a 
first and second three dimensional image and the first and second three dimensional images are 
combined to obtain the three dimensional image. 
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34. A method as in claim 30 further comprising determining a correspondence 
between the addresses in the two dimensional image and the addresses in the three dimensional 
image by calibrating to a machined fixture. 



35. A method as in claim 34 further comprising transforming the three dimensional 
image to remove geometric distortion. 

36. A method for evaluating the quality of an IC package where the IC package 
includes a plurality of three dimensional features, the method comprising: 

acquiring a two dimensional image characteristic of a portion of the IC package, 
the two dimensional image defined by a plurality of pixels having at least an address and a pixel 
intensity; 

acquiring a three dimensional image characteristic of the portion of the IC 
package, the three dimensional image defined by a plurality of pixels having at least an address 
and an altitude; 

processing the two dimensional image to identify a plurality of addresses which are 
characteristic three dimensional features; 

determining a correspondence between the addresses in the two dimensional image 
and the addresses in the three dimensional image by calibrating to a machined fixture. 

processing the three dimensional image only at those addresses which correspond 
to two dimensional addresses characteristic of three dimensional features, to determine the 
altitude of those three dimensional features, and; 

rejecting the IC package if the altitude of the three dimensional features fall outside 
predetermined boundaries. 

37. A method as in claim 36 wherein the processing of the three dimensional image 
further comprises determining the shape of the three dimensional features and the IC package is 
rejected if the shape of the three dimensional feature falls outside a predetermined boundary. 
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38. A method as in claim 37 further comprising comparing the two dimensional 
image against a two dimensional template and rejecting the quality of the IC package if the 
comparison reveals that the two dimensional image does not include three dimensional features in 
an expected configuration. 



39. A method as in claim 38 wherein the two dimensional image is a gray scale 
image and the two dimensional image is correlated against the two dimensional template. 

40. A method as in claim 39 wherein a pair of opposed lasers are used to obtain a 
first and second three dimensional image and the first and second three dimensional images are 
combined to obtain the three dimensional image. 



41 . A method as in claim 40 further comprising transforming the three dimensional 
image to remove geometric distortion. 
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